sAIfer Lab works on testing and improving AI security against adversarial attacks.

Within 2012-2013, Pra Lab pioneered the field of AI security and Adversarial Machine Learning (ML), demonstrating that AI/ML models can be misled by carefully-crafted attacks against them. In particular, we have been the first to demonstrate gradient-based evasion and poisoning attacks against AI/ML systems, also in the context of real-world applications, including malware and spam detection.

Evasion attacks subtly manipulate input data to deceive AI models, steering them towards making incorrect decisions. For example, a carefully crafted sticker placed on a stop sign can cause a state-of-the-art image recognition system to misclassify it as a yield sign, potentially leading to dire consequences in autonomous driving systems.

Poisoning attacks contaminate the training data of AI models to influence their behavior when the model is deployed. This type of attack can be particularly damaging, as it compromises the model's integrity and availability.

Our work, carried on by Pra Lab and SmartLab both, has significant implications across various domains, including:

• Computer Vision: enhancing the security of image recognition systems used in surveillance, biometrics, and autonomous vehicles.

• Robotics: developing AI models for robots that need to update their system to new unseen scenarios, e.g., when they need to classify new objects not seen during their training phase.

• Cybersecurity: strengthening AI-based security systems to protect against sophisticated cyber threats targeting the AI components, e.g., systems for detecting malicious software (malware), spam emails, and phishing attempts.
  
  

Our researchers have developed different tools that help to evaluate the impact of these attacks against ML systems. These tools include:

• SecML-Torch, a PyTorch-powered Python library to assess the security evaluation of AI/ML technologies against evasion and poisoning attacks;

• an extension of the previous library, called SecML Malware, ad-hoc for attacking Windows malware detectors;

• AttackBench, a benchmark framework for fairly comparing gradient-based attacks and developed to identify the most reliable one to use for robustness verification. 

To involve the academic and industrial community, we organize the MLSec Seminars, a series of events in which we invite researchers to talk about innovations and recent advancements in Machine Learning security.